Chapter 11 Classification

Classification is the problem of predicting the class of a given instance, on the basis of some attributes (features) of it. In the Bayesian framework 1, a classifier is learned from data by updating a prior density, which represents the beliefs before analyzing the data and which is usually assumed uniform, with the likelihood, which models the evidence coming from the data; this yields a posterior joint probability over classes and features. Once the classifier has been learned from data, it can classify novel instances; under 0-1 loss, it returns the most probable class after conditioning on the value of the features of the instance to be classified. Yet, Bayesian classifiers can happen to issue prior-dependent classifications, namely the most probable class changes under different priors. This might be acceptable if the prior has been carefully elicited and represents domain knowledge; however, in general the uniform prior is taken as default, without further investigation. This consideration have lead to the development of credal classifiers, which extend Bayesian classifiers to imprecise probabilities. The term “credal classifier” was firstly used in [685], when introducing the naive credal classifier (NCC). NCC generalizes naive Bayes, representing a condition of prior ignorance2 through the Imprecise Dirichlet Model (IDM) [643], namely adopting a set of priors instead of a single prior3. The set of priors is then updated by element-wise application of Bayes’ rule, yielding a set of posteriors. From this starting idea, imprecise-probability classification has been developed following two main directions: (a) extensions to imprecise probabilities of models based on Bayesian networks classifiers and (b) extension to imprecise probabilities of models based on classification trees. The common trait of such approaches is that they return a set of classes rather than a single class when there is not enough evidence for safely